Papers with two-stage framework
Bridging the Gap between Pre-Training and Fine-Tuning for Commonsense Generation (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods focusing on this task usually concatenate the concatened concepts words as the inputs of a pre-trained language model (PLM) however, in pre-training, the input is often corrupted sentences with correct word order. |
| Approach: | They propose a two-stage framework to improve the ability of pre-trained language models to deal with masked sentences with incorrect word order and a special token to make the input distribution more similar to the one used in pre-training. |
| Outcome: | The proposed method is able to generate a sentence containing all given concepts and correctly describe the relations between concepts. |
Explanation Selection Using Unlabeled Data for Chain-of-Thought Prompting (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent work shows how to prompt large language models with explanations to obtain strong performance on textual reasoning tasks. |
| Approach: | They propose to optimize explanation-infused prompts in a blackbox fashion by using leave-one-out schemes and a two-stage framework. |
| Outcome: | The proposed method improves prompts over crowdworker annotations and naive search strategies. |
Section-Aware Commonsense Knowledge-Grounded Dialogue Generation with Pre-trained Language Model (2022.coling-1)
Copied to clipboard
| Challenge: | Pre-trained language models can be expected to deepen the fusing of dialogue context and knowledge because of their superior ability of semantic understanding. |
| Approach: | They propose a two-stage framework to integrate a linearized knowledge into plan text using a ranking network PriorRanking to estimate the relevance of a retrieved knowledge fact. |
| Outcome: | The proposed framework improves the performance of pre-trained language models by using section-aware strategies to encode the linearized knowledge. |
Unsupervised Multilingual Dense Retrieval via Generative Pseudo Labeling (2024.findings-eacl)
Copied to clipboard
| Challenge: | Existing sparse retrieval methods often yield inferior performance in multilingual retrieval, requiring a large amount of paired data, which is costly. |
| Approach: | They propose an Unsupervised Multilingual dense Retriever trained without paired data which iteratively improves performance of multilingual retrievers. |
| Outcome: | The proposed framework outperforms supervised baselines on two benchmark datasets and shows that iterative training improves the performance. |
MindRef: Mimicking Human Memory for Hierarchical Reference Retrieval with Fine-Grained Location Awareness (2025.acl-short)
Copied to clipboard
| Challenge: | Existing methods require pre-segmented article chunks, limiting reference flexibility like human memory. |
| Approach: | They propose a framework that leverages parameterized knowledge stored during the pre-training phase of large language models to recall reference passages from any starting position independently. |
| Outcome: | The proposed framework can recall reference passages from any starting position independently. |
Improving Unsupervised Out-of-domain detection through Pseudo Labeling and Learning (2023.findings-eacl)
Copied to clipboard
| Challenge: | Unsupervised OOD detection is a task aimed at discriminating whether given samples are from the in-domain (IND) . previous studies adopted the one-class classification approach, assuming that the training samples come from a single domain. |
| Approach: | They propose a framework that leverages latent categorical information to improve representation learning for textual OOD detection. |
| Outcome: | The proposed framework significantly outperforms baseline models on three datasets. |
Bridging Modality Gap for Effective Multimodal Sentiment Analysis in Fashion-related Social Media (2025.coling-main)
Copied to clipboard
| Challenge: | Existing sentiment analysis tasks focus on text comprehension, but visual content is important for emotional expression. |
| Approach: | They propose a multimodal framework that integrates information from various modalities for sentiment classification of fashion posts. |
| Outcome: | The proposed framework outperforms existing unimodal and multimodal baselines on a comprehensive dataset and significantly outperformed existing unilmodal and multiple modal frameworks. |
ComfyUI-R1: Exploring Reasoning Models for Workflow Generation (2026.findings-acl)
Copied to clipboard
| Challenge: | ComfyUI-R1 is the first large reasoning model for automated workflow generation. |
| Approach: | They propose a large reasoning model for automated workflow generation that builds on curated knowledge bases and a two-stage framework to fine-tune models for cold start and reinforcement learning for incentivizing reasoning capability. |
| Outcome: | The proposed model achieves 97% format validity rate, high pass rate, node-level and graph-level F1 scores, surpassing prior state-of-the-art methods that employ leading closed-source models such as GPT-4o and Claude series. |
Learn and Review: Enhancing Continual Named Entity Recognition via Reviewing Synthetic Samples (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for named entity recognition classify mentions into fixed set of predefined entity types but in many real-world scenarios, new entity types are incrementally involved. |
| Approach: | They propose a two-stage framework Learn-and-Review for continual named entity recognition to alleviate inter-type confusion. |
| Outcome: | The proposed framework outperforms the state-of-the-art method on CoNLL-03 and OntoNotes-5.0. |
GoT: Effective Graph-of-Thought Reasoning in Language Models (2024.findings-naacl)
Copied to clipboard
| Challenge: | Recent advances in Large Language Models (LLMs) have been advancing at an unprecedented pace. |
| Approach: | They propose a graph-based approach which models human thought processes as a chain and as 'graphs' by representing thought units as nodes and connections between them as edges, they capture the non-sequential nature of human thinking and allows for a more realistic modeling of thought processes. |
| Outcome: | The proposed model improves on a text-only reasoning task and a multimodal reasoning task. |
AutoTriggER: Label-Efficient and Robust Named Entity Recognition with Auxiliary Trigger Extraction (2023.eacl-main)
Copied to clipboard
Dong-Ho Lee, Ravi Kiran Selvam, Sheikh Muhammad Sarwar, Bill Yuchen Lin, Fred Morstatter, Jay Pujara, Elizabeth Boschee, James Allan, Xiang Ren
| Challenge: | Named entity recognition models have shown impressive results in overcoming label scarcity and generalizing to unseen entities by leveraging distant supervision and auxiliary information such as explanations. |
| Approach: | They propose a framework that automatically generates and leverages “entity triggers” which are human-readable cues in the text that help guide the model to make better decisions. |
| Outcome: | The proposed framework outperforms the RoBERTa-CRF baseline by nearly 0.5 F1 points on three well-studied datasets. |
Extracting and Encoding: Leveraging Large Language Models and Medical Knowledge to Enhance Radiological Text Representation (2024.findings-acl)
Copied to clipboard
| Challenge: | Advancing representation learning in specialized fields like medicine remains challenging due to the scarcity of expert annotations for text and images. |
| Approach: | They propose a Fact Extractor that leverages large language models to extract factual statements from radiology reports. |
| Outcome: | The proposed framework outperforms current state-of-the-art methods in sentence ranking, natural language inference, and label extraction tasks. |
Exploring the Potential of Large Language Models for Heterophilic Graphs (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing approaches for heterophilic graphs overlook rich textual data associated with nodes, which could unlock deeper insights into their heterophilistic contexts. |
| Approach: | They propose a two-stage framework to enhance node classification on heterophilic graphs by leveraging open-world knowledge encoded by large language models. |
| Outcome: | The proposed framework can be used to better characterize heterophilic graphs, where neighboring nodes often exhibit different labels. |
Randomized Smoothing with Masked Inference for Adversarially Robust Text Classifications (2023.acl-long)
Copied to clipboard
| Challenge: | Large-scale pre-trained language models are brittle against specifically crafted adversarial examples, leading to increasing interest in probing the adversariality of NLP systems. |
| Approach: | They propose a two-stage framework that combines randomized smoothing and masked inference to improve the adversarial robustness of NLP systems. |
| Outcome: | The proposed framework improves adversarial robustness by 2 to 3 times over existing state-of-the-art methods on benchmark datasets. |
LLM-Driven Implicit Target Augmentation and Fine-Grained Contextual Modeling for Zero-Shot and Few-Shot Stance Detection (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies on zero-shot and few-shot stance detection neglect implicit yet semantically important targets. |
| Approach: | They propose a framework that uses Large Language Models to annotate implicit targets . they also propose 'DyMCA' to dynamically adjust text-target contributions based on context . |
| Outcome: | The proposed framework achieves state-of-the-art on a benchmark dataset. |
Leaner and Faster: Two-Stage Model Compression for Lightweight Text-Image Retrieval (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing text-image approaches use pre-trained vision-language representations for text retrieval . however, these models pose non-trivial memory requirements and substantial indexing time . |
| Approach: | They propose a framework to compress large pre-trained dual-encoders for lightweight text-image retrieval. |
| Outcome: | The proposed model performs better on Flickr30K and MSCOCO benchmarks than the original full model on mobile devices. |
Modeling Diagnostic Label Correlation for Automatic ICD Coding (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing work built a binary prediction for each label independently, ignoring the dependencies between labels. |
| Approach: | They propose a framework to capture the label correlation and train a reranking estimator to rescore the probability of each label set candidate generated by a base predictor. |
| Outcome: | The proposed framework improves on the best-performing predictors on MIMIC datasets. |
LSTDial: Enhancing Dialogue Generation via Long- and Short-Term Measurement Feedback (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing dialogue systems do not utilize quality dimensions specifically designed for dialogue evaluation to guide the response generation during training. |
| Approach: | They propose a two-stage framework which generates and utilizes conversation evaluation as explicit feedback during training. |
| Outcome: | The proposed framework generates and utilizes conversation evaluation as explicit feedback during training. |
Handling Missing Entities in Zero-Shot Named Entity Recognition: Integrated Recall and Retrieval Augmentation (2025.naacl-long)
Copied to clipboard
| Challenge: | Zero-shot Named Entity Recognition (ZS-NER) aims to recognize entities in unseen domains without specific annotated data. |
| Approach: | They propose a novel two-stage framework leveraging large language model techniques to improve the ZS-NER’s recall rate. |
| Outcome: | The proposed framework improves the ZS-NER’s recall rate and accuracy by incorporating a large language model. |
Character-centric Story Visualization via Visual Planning and Token Alignment (2022.emnlp-main)
Copied to clipboard
| Challenge: | Story visualization is a task that requires machines to understand long text inputs and produce a globally consistent image sequence that illustrates the contents of the story. |
| Approach: | They propose to augment VQ-VAE with a text-to-visual-token (transformer) architecture to enable multiple image generation based on a complete story. |
| Outcome: | The proposed method excels at preserving characters and produces higher quality image sequences compared with baselines. |
DisGeM: Distractor Generation for Multiple Choice Questions with Span Masking (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Multiple-choice cloze tests are a prevalent form of assessment that evaluates students' comprehension and inference abilities. |
| Approach: | They propose a framework for distractor generation using readily available pre-trained language models . human evaluations confirm that their approach produces more effective distractors . |
| Outcome: | The proposed framework outperforms existing methods without training or fine-tuning human evaluations confirm it. |
Aligning Backchannel and Dialogue Context Representations via Contrastive LLM Fine-Tuning (2026.acl-long)
Copied to clipboard
| Challenge: | Prior work on predicting backchannel timing has focused on lexical form and prosody, but the relationship between lexico-prosodic form and meaning remains underexplored. |
| Approach: | They propose a framework for fine-tuning large language models on dialogue transcripts to derive rich contextual representations; and a joint embedding space for dialogue contexts and backchannel realizations. |
| Outcome: | The proposed framework improves context-backchannel retrieval and human perception is more sensitive to extended conversational context and embeddings align more closely with human judgments than raw WavLM features. |
Self-supervised Quantized Representation for Seamlessly Integrating Knowledge Graphs with Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are gaining popularity due to their lack of knowledge hallucination and lack of a coherent model. |
| Approach: | They propose a self-supervised quantized representation method to compress KG structural and semantic knowledge into discrete codes that align the format of language sentences. |
| Outcome: | The proposed framework outperforms existing unsupervised methods producing more distinguishable codes on KG link prediction and triple classification tasks. |
CARE: A Disagreement Detection Framework with Concept Alignment and Reasoning Enhancement (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to disagreement detection are limited by conceptual gap and reasoning gap. |
| Approach: | They propose a conceptual alignment and reasoning enhancement framework to address the conceptual gap and the reasoning gap in disagreement detection. |
| Outcome: | The proposed framework shows superior performance in zero-shot and supervised learning settings, both within and across domains. |
Disambiguated Lexically Constrained Neural Machine Translation (2023.findings-acl)
Copied to clipboard
| Challenge: | Current approaches to LCNMT assume that pre-specified lexicon constraints are contextually appropriate. |
| Approach: | They propose a framework that disambiguates constraints based on contexts at first and integrates them into LCNMT. |
| Outcome: | The proposed approach outperforms baseline approaches on benchmark datasets and comprehensive experiments in multiple target constraints. |
How to Mitigate Overfitting in Weak-to-strong Generalization? (2025.acl-long)
Copied to clipboard
| Challenge: | Experimental results show that weak-to-strong generalization significantly improves PGR compared to naive weak- to-strong . superalignment refers to how humans can align models on tasks beyond human ability to evaluate . |
| Approach: | They propose a framework that elicits the capabilities of strong models through weak supervisors . they propose 'superalignment' to ensure that strong models align with supervisors' intentions . |
| Outcome: | The proposed framework significantly improves quality of supervision signals and quality of input questions compared to naive weak-to-strong generalization . |
Activation-Guided Local Editing for Jailbreaking Attacks (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for jailbreaking Large Language Models (LLMs) are limited and produce incoherent or unreadable inputs. |
| Approach: | They propose a two-stage framework that performs a one-shot, scenario-based generation of context and rephrases the original malicious query to obscure its harmful intent. |
| Outcome: | The proposed framework achieves state-of-the-art Attack Success Rate, with gains of up to 37.74% over the strongest baseline, and excellent transferability to black-box and large-scale models. |
ClinAlign: Scaling Healthcare Alignment from Clinician Preference (2026.findings-acl)
Copied to clipboard
Shiwei Lyu, Xidong Wang, Hao Zhu, Lei Liu, Chaohe Zhang, Jian Wang, Jinjie Gu, Benyou Wang, Yue Shen
| Challenge: | Existing methods for aligning open-ended outputs with fine-grained clinician preferences are weakly grounded in professional guidelines. |
| Approach: | They propose a framework to align large language models' outputs with fine-grained clinician preferences . they propose 119 broadly reusable, clinically grounded principles organized by clinical dimensions . |
| Outcome: | The proposed framework outperforms existing models on HealthBench-Hard and Deepseek-R1 and o3. |
Chain of Strategy Optimization Makes Large Language Models Better Emotional Supporter (2025.findings-emnlp)
Copied to clipboard
Weixiang Zhao, Xingyu Sui, Xinyang Han, Yang Deng, Yulin Hu, Jiahe Guo, Libo Qin, Qianyun Du, Shijin Wang, Yanyan Zhao, Bing Qin, Ting Liu
| Challenge: | Existing supervised fine-tuning (SFT) fails to address these issues, as it trains models on single gold-standard responses without modeling nuanced strategy trade-offs. |
| Approach: | They propose a two-stage framework that optimizes strategy selection preferences at each dialogue turn. |
| Outcome: | The proposed framework improves strategy selection preferences at each dialogue turn. |
A Facial Expression-Aware Multimodal Multi-task Learning Framework for Emotion Recognition in Multi-party Conversations (2023.acl-long)
Copied to clipboard
| Challenge: | Recent studies have shown the importance of visual information in multi-party conversations due to the complexity of visual scenes. |
| Approach: | They propose a framework to extract face sequences as visual features from a real speaker's utterance and a pipeline method to extract the face sequence. |
| Outcome: | The proposed framework extracts face sequences of the real speaker of each utterance and improves emotion prediction on the MELD dataset. |
Learning from Cognition: Enhancing RL Efficiency for LLM Reasoning via Hierarchical Metacognitive Decomposition and Refinement (2026.acl-long)
Copied to clipboard
Zexu Sun, Yongcheng Zeng, Erxue Min, Heyang Gao, Bokai Ji, Dugang Liu, Xing Tang, Xiuqiang He, Xu Chen
| Challenge: | Recent advances in Large Language Models have demonstrated notable inferential capacities via reinforcement learning (RL) however, “zero-RL” approaches relying on fixed prompt templates introduce substantial sampling inefficiencies for weak LLMs. |
| Approach: | They propose a hierarchical metacognitive RL framework that decomposes zero-accuracy problems into subproblems and prompts the policy to refine answers by referencing previous wrong solutions. |
| Outcome: | The proposed framework improves sample utilization and sample efficiency and accelerates convergence compared to baselines. |
Post Persona Alignment for Multi-Session Dialogue Generation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for multi-session persona-based dialogue generation typically retrieve persona information before response generation, which can constrain diversity and result in generic outputs. |
| Approach: | They propose a two-stage framework that reverses the process of retrieving persona information before response generation. |
| Outcome: | Experiments on multi-session persona-based dialogue data show that the proposed framework outperforms existing methods in consistency, diversity, and persona relevance. |
Thoughts to Target: Enhance Planning for Target-driven Conversation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Empirical results demonstrate that our method significantly improves the planning ability of LLMs, especially in target-driven conversations. |
| Approach: | They propose a two-stage framework to improve the LLMs’ capability in planning conversations towards designated targets by distilling natural language plans from a target-driven conversation corpus and generating new plans with demonstration-guided in-context learning. |
| Outcome: | The proposed framework improves the ability of conversational models to plan towards designated targets and can be used to build extensive conversational AI. |
Sculpting the Vector Space: Towards Efficient Multi-Vector Visual Document Retrieval via Prune-then-Merge Framework (2026.findings-acl)
Copied to clipboard
| Challenge: | Visual Document Retrieval (VDR) is of importance in multimodal retrieval applications. |
| Approach: | They propose a two-stage pruning and merging frameworks that combine pruning and merge techniques to achieve higher compression rates. |
| Outcome: | The proposed framework outperforms existing methods on 29 visual document retrieval datasets. |
Counterspeech the ultimate shield! Multi-Conditioned Counterspeech Generation through Attributed Prefix Learning (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods to generate counterspeech based on intents are limited to single attributed . however, a holistic approach that considers multiple attributes simultaneously yields more nuanced and effective responses. |
| Approach: | They propose a framework that leverages hierarchical prefix learning with preference optimization to generate more constructive counterspeech. |
| Outcome: | The proposed framework improves intent conformity and emotion labels in 13,973 counterspeech instances. |
Do Vision-Language Models Have Internal World Models? Towards an Atomic Evaluation (2025.findings-acl)
Copied to clipboard
Qiyue Gao, Xinyu Pi, Kevin Liu, Junrong Chen, Ruolan Yang, Xinqi Huang, Xinyu Fang, Lu Sun, Gautham Kishore, Bo Ai, Stone Tao, Mengyang Liu, Jiaxi Yang, Chao-Jung Lai, Chuanyang Jin, Jiannan Xiang, Benhao Huang, Zeming Chen, David Danks, Hao Su, Tianmin Shu, Ziqiao Ma, Lianhui Qin, Zhiting Hu
| Challenge: | Recent studies have evaluated and shown limitations in specific capabilities such as visual understanding, but a systematic evaluation of VLMs’ fundamental WM abilities remains absent. |
| Approach: | They propose a framework that assesses perception and prediction to provide an atomic evaluation of VLMs as WMs. |
| Outcome: | The proposed framework assesses perception and prediction abilities on 15 latest VLMs and compares them to human-level models. |
CodeRise: Bootstrapping LLMs for Ultra Low-Resource Programming Languages via Progressive Self-Refinement Curriculum (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for training data generation for low-resource languages suffer from a cold-start problem and lack diversity. |
| Approach: | They propose a two-stage framework that generates a high-quality, diverse, and progressively complex curriculum for Ultra Low-Resource Programming Languages (ULRPLs) they leverage the full formal syntax of the target language as structural guidance and apply a biased sampling strategy over library modules. |
| Outcome: | The proposed framework outperforms training-free and training-based baselines on two ULRPLs, Tengo and Janet. |
When in Doubt, Consult: Expert Debate for Sexism Detection via Confidence-Based Routing (2026.acl-long)
Copied to clipboard
| Challenge: | sexist content on social media is increasingly pervasive, often appearing in subtle, context-dependent forms that evade traditional classification methods. |
| Approach: | They propose a framework that unifies targeted training procedures to regularize supervision to scarce and noisy data with selective reasoning-based inference to handle ambiguous or borderline cases. |
| Outcome: | The proposed framework outperforms existing approaches across several public benchmarks . it bridges the gap between efficiency and reasoning with a dynamic routing mechanism . |
Prompt Optimization for Relation Extraction using Reinforcement Learning (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing prompt-based methods rely heavily on large-scale annotated datasets limiting their applicability in domain-specific and low-resource scenarios. |
| Approach: | They propose a reinforcement learning-based automated prompt optimization framework for domain relation extraction that optimizes prompt quality through interaction with a black-box LLM. |
| Outcome: | The proposed framework outperforms existing prompt-based methods and supervised baselines on multiple extraction datasets across medical, financial, legal, and news domains. |